Generation of idioms in a simple recurrent network architecture

Idioms are an ideal testbed for studying the interplay of lexical (content preparing) and syntactic (structure building) mechanisms in language production. This article contributes to the debate about the nature of these mechanisms and their relationship from the viewpoint of computational modeling. We present a neural network model of sentence generation, which is able to produce continuous and discontinuous idioms within regular compositional sentences. The model is a simple recurrent network extended to include a semantic episode representation as an extra input. Our main contribution consists in a detailed analysis of the representational space of the network’s hidden layer, which shows that (1) an implicit structure-content division can arise as a result of internal space reorganization within a single SRN during learning, (2) idioms can be produced by the same general sequencing mechanism that works for regular sentences, (3) the production of idioms is modulated by content-specific mechanisms.